Field of the Invention
The present invention relates to a processing apparatus, a processing system, an image pickup apparatus, a processing method, and a non-transitory computer-readable storage medium.
Description of the Related Art
Obtaining more physical information regarding an object can generate images based on a physical model in image processing after imaging. For example, an image where visibility of the object is changed can be generated. Visibility of the object is determined on the basis of information such as shape information of the object, reflectance information of the object and light source information. As physical behavior of reflected light that is emitted from a light source and is reflected by the object depends on a local surface normal, using not a three-dimensional shape but surface normal information of the object as shape information is especially effective.
A photometric stereo method, in which reflectance characteristics of the object based on the surface normal of the object and a light source direction is assumed and the surface normal is calculated from luminance information of the object at a plurality of light source positions and the assumed reflectance characteristics, has been known (refer to “Photometric stereo” by Yasuyuki Matsushita, A research report of Information Processing Society of Japan, Vol. 2011-CVIM-177, No. 29, pp. 1-12, 2011). A Lambert reflection model in dependence upon a Lambert's cosine law is often used as the reflectance characteristics of the object.
In general, reflected light of the object has each component of specular reflected light and diffusion reflected light. The specular reflected light is regular reflection on an object surface, and indicates Fresnel reflection in dependence upon a Fresnel equation on the object surface (interface). The diffusion reflection light indicates light which is diffused into the object after transmitting through the object surface. As a specular reflection component is not expressed by the Lambert's cosine law, when reflected light from the object measured by an image pickup apparatus includes the specular reflection component, the surface normal cannot be calculated using the photometric stereo method accurately. Moreover, in shading parts which are not exposed to light from a light source, as deviation from the assumed reflection model occurs, the surface normal information of the object cannot be obtained accurately. For example, in Japanese Patent Laid-Open 2010-122158, a method to calculate a real surface normal from a plurality of surface normal candidates, which are obtained using four or more light sources, is disclosed.
To obtain the surface normal of the object using the photometric stereo method in the image pickup apparatus such as a digital camera, a plurality of images having different luminance information for each irradiation light source position are required. In the photometric stereo method, as it is assumed that a light quantity to be irradiated on the object is the same, when differences are generated in the irradiation light quantity among a plurality of light sources having different positions, the calculated surface normal largely deviates from the real surface normal. In “Photometric stereo” by Yasuyuki Matsushita, A research report of Information Processing Society of Japan, Vol. 2011-CVIM-177, No. 29, pp. 1-12, 2011 and Japanese Patent Laid-Open 2010-122158, response to influence due to the differences of the irradiation light quantity is not disclosed, and when the irradiation light quantity fluctuates for some reason, calculation accuracy of the surface normal deteriorates.